{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IO编程\n",
    "1. 同步IO\n",
    "2. 异步IO：回调模式；轮询模式\n",
    "\n",
    "\n",
    "- 基本概念：input， output，stream\n",
    "- 存在问题：输入和接收速度不匹配\n",
    "- 解决方法：同步、异步(回调--好了叫我，轮询---好了没...好了没)\n",
    "- 收获新知：编程语言都会把操作系统提供的低级C接口封装起来方便使用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 文件读写"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 读写文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.文件对象: <_io.TextIOWrapper name='./Log/test_file.txt' mode='a+' encoding='cp936'> <class '_io.TextIOWrapper'>\n",
      "2.文件的当前内容：\n",
      "Hello World!\n",
      "\n",
      "Hello Python.\n",
      "\n",
      "Where there is a will, there is a way.007Hi\n",
      "\n",
      "3.写入新的数据:\n",
      "['Hello World!\\n', 'Hello Python.\\n', 'Where there is a will, there is a way.007Hi\\n']\n"
     ]
    }
   ],
   "source": [
    "# 使用open()函数读取文件\n",
    "# 列出该文件夹下的内容\n",
    "# !dir .\\\\LOG   \n",
    "f = open('./Log/test_file.txt', 'a+')\n",
    "f.seek(0)\n",
    "print('1.文件对象:', f, type(f))\n",
    "print('2.文件的当前内容：')\n",
    "for line in f:         # 迭代访问\n",
    "    print(line)\n",
    "f.seek(0)              # 因为上面的迭代文件指针已经到末尾了，所以回到文件头便于再次访问\n",
    "print('3.写入新的数据:')\n",
    "out = f.writelines(['0','07', 'Hi\\n'])    # 写入数据，以字符列表输入即可，写入字符间无间隔\n",
    "# 由于这里没有关闭文件并重新读入所以比实际文件少一次添加数据的结果\n",
    "print(f.readlines())                    # readlines()以列表形式返回所有数据，每行一个元素\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.常用的文件打开模式\n",
    "\n",
    "\n",
    "| 模式 | 描述 |\n",
    "|- |- |\n",
    "|r    |只读，文件指针放在文件开头，默认模式|\n",
    "|w|写入，文件指针放在文件开头|\n",
    "|a|追加写入，文件指针放在文件末尾|\n",
    "|rb wb ab|以二进制的形式操作文件|\n",
    "|+|上述模式末尾有该符号，都变成可读写|\n",
    "\n",
    "2.file对象的操作方法\n",
    "\n",
    "\n",
    "- **file.read([size])** size未指定则返回整个文件，读取到文件尾时返回空字符串\n",
    "- **file.readline()** 返回一行的数据\n",
    "- **file.readlines([size])** 返回包含size行的列表，未指定则返回全部行\n",
    "- **for line in file:** 通过迭代器方法\n",
    "- **file.write('hello\\n')**  写入一行字符串，参数必须是字符串\n",
    "- **file.writelines('seq\\n')** 写入字符串，可以输入元素为字符串的列表，换行用\\n来控制\n",
    "- **file.tell()** 返回文件指针的位置(文件头的比特数)\n",
    "- **file.seek(偏移量[,起始位置])** 移动文件指针，偏移量可正可负；起始位置：0-文件头，1-当前位置，2-文件尾\n",
    "- **file.close()** 关闭文件，否则文件对象会占用操作系统资源"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用try...finally来保证文件正确地关闭"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IOError: [Errno 2] No such file or directory: 'LOG/test_file_noexists.txt'\n"
     ]
    }
   ],
   "source": [
    "# 1.try...finally实现\n",
    "def file_open(file_name):\n",
    "    try:\n",
    "        temp_file = ''                     # 保证文件不存在时，该变量的引用部分正常工作\n",
    "        temp_file = open(file_name, 'r')   # 打开只读文件\n",
    "        print(temp_file.read())\n",
    "    except Exception as err:\n",
    "        print('IOError:', err)\n",
    "#         raise                      # 将异常继续抛出，可以提供更多的错误信息\n",
    "    finally:\n",
    "        if temp_file:\n",
    "            temp_file.close()\n",
    "file_open('LOG/test_file_noexists.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用with语句来自动调用close()函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello World!\n",
      "Hello Python.\n",
      "Where there is a will, there is a way.007Hi\n",
      "007Hi\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 2. with语句可以自动关闭文件对象，更好用\n",
    "def file_open2(file_name):\n",
    "    with open(file_name, 'r') as temp_file:\n",
    "        print(temp_file.read())   \n",
    "file_open2('LOG/test_file.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.直接调用文件对象:\n",
      "Hello World!\n",
      "\n",
      "Hello Python.\n",
      "\n",
      "Where there is a will, there is a way.007Hi\n",
      "\n",
      "007Hi\n",
      "\n",
      "2.使用readlines函数返回文件内容:\n",
      "Hello World!\n",
      "Hello Python.\n",
      "Where there is a will, there is a way.007Hi\n",
      "007Hi\n"
     ]
    }
   ],
   "source": [
    "f = open('./Log/test_file.txt', 'r')\n",
    "print('1.直接调用文件对象:')\n",
    "for line in f:\n",
    "    print(line)\n",
    "print('2.使用readlines函数返回文件内容:')\n",
    "f.seek(0)         # 回到文件头部\n",
    "for line in f.readlines():\n",
    "    print(line.strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. file-like Object\n",
    "一个对象只要有read方法，就可以调用open()函数，该对象也不必从特点的类继承，所以file、字节流、网络流和自定义流都可以使用open()函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在**3_面向对象的类**笔记中有用到\"鸭子类型\"，可复习该部分了解file-like Object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 二进制文件"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "文本文件一般为UTF-8编码，open()函数可以读取二进制文件，比如图片、视频等"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.显示图像的16进制字节结果(部分): b'\\xff\\xd8\\xff\\xe0\\x00\\x10JFIF\\x00\\x01\\x01\\x01\\x00`\\x00`\\x00\\x00' 20\n"
     ]
    }
   ],
   "source": [
    "f = open('test.jpg', 'rb')   # 以二进制打开图片\n",
    "out = f.read()[:20]   # 会显示16进制的字节结果\n",
    "print('1.显示图像的16进制字节结果(部分):', out, len(out))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取非UTF-8编码，传入enoding参数，ignore参数忽略非法编码字符\n",
    "# f = open(file_name, 'r', encoding='gbk', errors='ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 StringIO和BytesIO\n",
    "在内存中读取数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1) StringIO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.stream position: 0\n",
      "  stream position: 5\n",
      "  stream position: 12\n",
      "2.输出的结果:\n",
      "hello world!\n",
      "hello world!\n"
     ]
    }
   ],
   "source": [
    "# 1.写入str流，把string写入StrinigIO(内存中的数据)\n",
    "from io import StringIO\n",
    "f = StringIO()        # 实例化一个StringIO的实例\n",
    "print('1.stream position:', f.tell())\n",
    "f.write('hello')\n",
    "print('  stream position:', f.tell())\n",
    "f.write(' world!')\n",
    "print('  stream position:', f.tell())    # 指向的位置和字符串大小对应\n",
    "print('2.输出的结果:')\n",
    "print(f.getvalue())                      # getvalue获取写入的字符串，字符指针不用指到0就可以读取\n",
    "\n",
    "# print(f.getvalue())\n",
    "f.seek(0)\n",
    "print(f.readline())  # 字符指针必须指向0才可以读取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "Hello!\n",
      "Hi!\n",
      "Goodbye!\n"
     ]
    }
   ],
   "source": [
    "# 2. 读取str流\n",
    "f2 = StringIO('Hello!\\nHi!\\nGoodbye!')\n",
    "print(f2.tell())\n",
    "while True:\n",
    "    line = f2.readline()   # 读取一行的数据\n",
    "    if line == '':\n",
    "        break\n",
    "    print(line.strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2) BytesIO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'\\xe4\\xb8\\xad\\xe6\\x96\\x87'\n"
     ]
    }
   ],
   "source": [
    "# 1. 在内存中写入二进制数据\n",
    "from io import BytesIO\n",
    "f = BytesIO()           # 实例化一个二进制流的实例\n",
    "f.write('中文'.encode('utf-8'))    # 写入的是二进制编码，bytes数据\n",
    "print(f.getvalue())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.进过编码的二进制数据(字节流): b'\\xe4\\xb8\\xad\\xe6\\x96\\x87'\n",
      "2.进过解码得到的数据: 中文\n"
     ]
    }
   ],
   "source": [
    "# 2. 在内存中读取数据\n",
    "f2 = BytesIO(b'\\xe4\\xb8\\xad\\xe6\\x96\\x87')\n",
    "out = f2.readline()\n",
    "# out = f2.getvalue()\n",
    "print('1.进过编码的二进制数据(字节流):', out)\n",
    "print('2.进过解码得到的数据:', out.decode('utf-8'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "小结:\n",
    "- file.readline()输出文件内容，文件指针必须在0的位置，使用file.seek(0)就行\n",
    "- file.getvalue()对文件指针位置没有要求"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 操作文件和目录"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "os模块：调用系统提供的接口，进行目录和文件的相关操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'nt'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "os.name   # 显示操作系统类型 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "default\n"
     ]
    }
   ],
   "source": [
    "# os.environ['path']     # 环境变量，两种方法都行，访问字典的方法\n",
    "print(os.environ.get('path2', 'default'))  # 没有该key，输出字符串default，默认输出None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.当前目录的绝对路径: C:\\Users\\David\\Desktop\\python基础巩固\\python_learning\n",
      "2.新目录的完整路径: C:\\Users\\David\\Desktop\\python基础巩固\\python_learning\\testdir\n"
     ]
    }
   ],
   "source": [
    "current_path = os.path.abspath('.')\n",
    "print('1.当前目录的绝对路径:', current_path)\n",
    "new_path = os.path.join(current_path, 'testdir')    # 本质上该函数进行字符串的处理，不同系统结果不同\n",
    "print('2.新目录的完整路径:', new_path)\n",
    "if not os.path.exists(new_path):     # 该目录不存在\n",
    "    os.mkdir(new_path)               # 创建该目录\n",
    "if os.path.exists(new_path):\n",
    "    os.rmdir(new_path)               # 删除该目录"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.新添加的路径: C:\\Users\\David\\Desktop\\python基础巩固\\python_learning\\test_dir\n",
      "2.拆分后的路径: ('C:\\\\Users\\\\David\\\\Desktop\\\\python基础巩固\\\\python_learning', 'test_dir')\n",
      "  使用字符串split函数分割的结果: ['C:', 'Users', 'David', 'Desktop', 'python基础巩固', 'python_learning', 'test_dir']\n",
      "3.新文件名称及路径: C:\\Users\\David\\Desktop\\python基础巩固\\python_learning\\test_dir\\test.txt\n",
      "  扩展名分割结果: ('C:\\\\Users\\\\David\\\\Desktop\\\\python基础巩固\\\\python_learning\\\\test_dir\\\\test', '.txt')\n"
     ]
    }
   ],
   "source": [
    "# 处理路径时使用os.path下的方法，尽量少用直接字符串连接，os.path可以使用系统\n",
    "# 1.添加路径\n",
    "new_path = os.path.join(current_path, 'test_dir')      # 可以连接多个字符串\n",
    "print('1.新添加的路径:', new_path)\n",
    "# 2.拆分路径，得到一个二元组，一般拆出最后级别的目录\n",
    "split_path = os.path.split(new_path)\n",
    "print('2.拆分后的路径:', split_path)\n",
    "# 拆分目录的第二个思路，使用split函数\n",
    "split_list = new_path.split('\\\\')\n",
    "print('  使用字符串split函数分割的结果:', split_list)\n",
    "# 3.得到文件的扩展名\n",
    "new_file = os.path.join(current_path, 'test_dir', 'test.txt')   # 创建新文件名称及路径\n",
    "print('3.新文件名称及路径:', new_file)\n",
    "extension_name = os.path.splitext(new_file)\n",
    "print('  扩展名分割结果:', extension_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_path = os.path.join(current_path, 'LOG', 'test_os2.txt')\n",
    "new_file_path = os.path.join(current_path, 'LOG', 'test_os_rename.txt')\n",
    "if os.path.exists(file_path):\n",
    "    if os.path.exists(new_file_path):\n",
    "        os.remove(new_file_path)             # 删除已经存在的新文件\n",
    "    os.rename(file_path, new_file_path)      # 给旧文件重命名\n",
    "if not os.path.exists(file_path):\n",
    "    with open(file_path, 'w') as file:       # 创建一个旧文件\n",
    "        file.writelines('Hello Python.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\David\\\\Desktop\\\\python基础巩固\\\\python_learning\\\\LOG\\\\test_os_copy.txt'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# shutil模块做文件、目录复制移动\n",
    "import shutil\n",
    "new_file_path = os.path.join(current_path, 'LOG', 'test_os_copy.txt')\n",
    "shutil.copyfile(file_path, new_file_path)   # 复制文件，两个参数必须都是文件名\n",
    "# shutil.copy(file_path, '.')                   # 复制文件,后者可以是目录\n",
    "# shutil.copytree('old_dir', 'new_dir')       # 复制目录及以下的所有的文件到新目录下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.当前目录下的所有目录: ['.git', '.ipynb_checkpoints', 'LOG', 'test_unit_document', '__pycache__']\n",
      "2.当前目录下的所有文件: ['1_python基础.ipynb', '2_python函数.ipynb', '3_面向对象的类.ipynb', '4_面向对象的高级编程.ipynb', '5_错误、调试和测试.ipynb', '6_IO编程.ipynb', 'Hello.py', 'README.md', 'test.jpg', 'test_model.py', '字典和生成器.ipynb']\n",
      "3.当前目录下所有notebook文件: ['1_python基础.ipynb', '2_python函数.ipynb', '3_面向对象的类.ipynb', '4_面向对象的高级编程.ipynb', '5_错误、调试和测试.ipynb', '6_IO编程.ipynb', '字典和生成器.ipynb']\n"
     ]
    }
   ],
   "source": [
    "# 列出当前的目录下所有符合要求的目录及文件\n",
    "out_list = [x for x in os.listdir('.') if os.path.isdir(x)]\n",
    "print('1.当前目录下的所有目录:', out_list)\n",
    "out_list = [x for x in os.listdir('.') if os.path.isfile(x)]\n",
    "print('2.当前目录下的所有文件:', out_list)\n",
    "# splitext得到二元组，第二个元素为文件扩展名\n",
    "out_list = [x for x in os.listdir('.') if os.path.isfile(x) and os.path.splitext(x)[1]=='.ipynb']\n",
    "print('3.当前目录下所有notebook文件:', out_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "文件和目录操作练习"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2018/08/10  10:08   <DIR>           .git\n",
      "2018/08/10  22:14   <DIR>           .ipynb_checkpoints\n",
      "2018/08/10  11:00           29628   1_python基础.ipynb\n",
      "2018/08/02  21:00           58502   2_python函数.ipynb\n",
      "2018/08/07  14:21           38737   3_面向对象的类.ipynb\n",
      "2018/08/08  22:32           32316   4_面向对象的高级编程.ipynb\n",
      "2018/08/10  21:49           16743   5_错误、调试和测试.ipynb\n",
      "2018/08/11  17:51           34379   6_IO编程.ipynb\n",
      "2018/08/10  14:21           721     Hello.py\n",
      "2018/08/11  17:51   <DIR>           LOG\n",
      "2018/08/02  01:09           148     README.md\n",
      "2018/08/10  12:44   <DIR>           __pycache__\n",
      "2018/08/10  16:21           93581   test.jpg\n",
      "2018/07/29  14:42           288     test_model.py\n",
      "2018/08/10  14:11   <DIR>           test_unit_document\n",
      "2018/07/04  11:44           13990   字典和生成器.ipynb\n"
     ]
    }
   ],
   "source": [
    "# 1.使用os模块函数，模拟系统dir的输出\n",
    "# 方法一\n",
    "import os\n",
    "import time\n",
    "def dir_l():\n",
    "    dir_list = os.listdir('.')\n",
    "    dir_list = sorted(dir_list)   # 将列表进行排序\n",
    "#     print(dir_list)\n",
    "    for content in dir_list:      # 通过列表，获取每个目录或文件的信息\n",
    "#         if os.path.isdir(content):  # 判断该文件是否是路径\n",
    "#             DIR = '<DIR>'\n",
    "#             file_size = ''\n",
    "#         else:\n",
    "#             DIR = ''\n",
    "#             file_size = os.stat(content). st_size\n",
    "        flag, file_size = ('<DIR>', '') if os.path.isdir(content) else ('', os.stat(content). st_size)\n",
    "        file_date = time.localtime(os.stat(content).st_mtime)\n",
    "        out_str = '{:}/{:0>2}/{:0>2}  {:0>2}:{:0>2}   {:<6}  {:<6}  {}'.format(file_date.tm_year, file_date.tm_mon, file_date.tm_mday,\\\n",
    "                                                   file_date.tm_hour, file_date.tm_min,\\\n",
    "                                                   flag, file_size, content)\n",
    "        print(out_str)\n",
    "    \n",
    "dir_l()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2018-08-10 10:08\t     <DIR>\t          \t.git      \n",
      "2018-08-10 22:14\t     <DIR>\t          \t.ipynb_checkpoints\n",
      "2018-08-10 11:00\t          \t     29628\t1_python基础.ipynb\n",
      "2018-08-02 21:00\t          \t     58502\t2_python函数.ipynb\n",
      "2018-08-07 14:21\t          \t     38737\t3_面向对象的类.ipynb\n",
      "2018-08-08 22:32\t          \t     32316\t4_面向对象的高级编程.ipynb\n",
      "2018-08-10 21:49\t          \t     16743\t5_错误、调试和测试.ipynb\n",
      "2018-08-11 17:51\t          \t     34379\t6_IO编程.ipynb\n",
      "2018-08-10 14:21\t          \t       721\tHello.py  \n",
      "2018-08-11 17:51\t     <DIR>\t          \tLOG       \n",
      "2018-08-02 01:09\t          \t       148\tREADME.md \n",
      "2018-08-10 16:21\t          \t     93581\ttest.jpg  \n",
      "2018-07-29 14:42\t          \t       288\ttest_model.py\n",
      "2018-08-10 14:11\t     <DIR>\t          \ttest_unit_document\n",
      "2018-08-10 12:44\t     <DIR>\t          \t__pycache__\n",
      "2018-07-04 11:44\t          \t     13990\t字典和生成器.ipynb\n"
     ]
    }
   ],
   "source": [
    "# 方法二\n",
    "import os\n",
    "import datetime\n",
    "for x in os.listdir('.'):\n",
    "#     size = os.path.getsize(x) if (not os.path.isdir(x)) else ''\n",
    "    mtime = datetime.datetime.fromtimestamp(os.path.getmtime(x)).strftime('%Y-%m-%d %H:%M')\n",
    "#     flag = '<DIR>' if os.path.isdir(x) else ''\n",
    "    flag, size = ('<DIR>', '') if  os.path.isdir(x) else ('', os.path.getsize(x))\n",
    "    print('%10s\\t%10s\\t%10s\\t%-10s' % (mtime, flag, size, x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".\\1_python基础.ipynb\n",
      ".\\2_python函数.ipynb\n",
      ".\\3_面向对象的类.ipynb\n",
      ".\\4_面向对象的高级编程.ipynb\n",
      ".\\5_错误、调试和测试.ipynb\n",
      ".\\6_IO编程.ipynb\n",
      "dir: .\\LOG\\LOG\n",
      "dir: .\\LOG\n",
      "dir: .\\test_unit_document\n",
      ".\\字典和生成器.ipynb\n"
     ]
    }
   ],
   "source": [
    "# 2.编写一个程序，能在当前目录以及当前目录的所有子目录下查找文件名包含指定字符串的文件，并打印出相对路径\n",
    "# 方法一、 先用递归方法试试\n",
    "import os\n",
    "string = 'ipynb'                           # 需要包含的字符串\n",
    "def find_file(current_path):\n",
    "    dir_list = os.listdir(current_path)    # 列出当前目录下的所有文件及文件夹\n",
    "    for content in dir_list:\n",
    "        if content[0] not in ('.', '_'):   # 排除. __开头的目录\n",
    "            content = os.path.join(current_path, content)          # 获取相对路径\n",
    "            if os.path.isfile(content) and (string in content):    # 找到相应的文件\n",
    "                print(content)\n",
    "            if os.path.isdir(content):     # 判断是目录，进行递归\n",
    "                find_file(content)\n",
    "                print('dir:', content)\n",
    "            \n",
    "find_file('.')   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".\\1_python基础.ipynb\n",
      ".\\2_python函数.ipynb\n",
      ".\\3_面向对象的类.ipynb\n",
      ".\\4_面向对象的高级编程.ipynb\n",
      ".\\5_错误、调试和测试.ipynb\n",
      ".\\6_IO编程.ipynb\n",
      ".\\字典和生成器.ipynb\n",
      ".\\.ipynb_checkpoints\\1_python基础-checkpoint.ipynb\n",
      ".\\.ipynb_checkpoints\\2_python函数-checkpoint.ipynb\n",
      ".\\.ipynb_checkpoints\\3_面向对象的类-checkpoint.ipynb\n",
      ".\\.ipynb_checkpoints\\4_面向对象的高级编程-checkpoint.ipynb\n",
      ".\\.ipynb_checkpoints\\5_错误、调试和测试-checkpoint.ipynb\n",
      ".\\.ipynb_checkpoints\\6_IO编程-checkpoint.ipynb\n"
     ]
    }
   ],
   "source": [
    "# 方法二\n",
    "def get_abspath(s):\n",
    "    path = '.'  # path可随指定路径不同而进行修改，在这表示当前路径\n",
    "    current_path = os.path.abspath('.')\n",
    "    for root, dir_names, file_names in os.walk(path):\n",
    "        for file_name in file_names:\n",
    "            if s in file_name:\n",
    "                print(os.path.abspath(os.path.join(root, file_name)).replace(current_path,'.'))\n",
    "    return\n",
    "get_abspath('ipynb')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "小结:\n",
    "- 在上面的递归方法中，一定要加入文件或文件夹的相对路径，否则isfile、isdir无法正常判断\n",
    "- os.walk(path_name)函数会遍历路径下所有的文件及文件夹\n",
    "- os.path.join split splitext方法本质是对字符串处理，与有无该文件及目录无关\n",
    "- os.path路径处理便于跨平台\n",
    "- [python读写、创建文件的方法](https://blog.csdn.net/lzc4869/article/details/76577761)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 序列化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "序列化：将内存中的变量和数据变为可存储和传输的过程 \n",
    "\n",
    "###  1. pickle模块提高了序列化和反序列化的方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'\\x80\\x03}q\\x00(X\\x04\\x00\\x00\\x00nameq\\x01X\\x03\\x00\\x00\\x00Bobq\\x02X\\x03\\x00\\x00\\x00ageq\\x03K\\x14X\\x05\\x00\\x00\\x00scoreq\\x04Kdu.'\n",
      "<class 'bytes'>\n"
     ]
    }
   ],
   "source": [
    "# 1. 将字典变量序列化存储到文件中\n",
    "import pickle\n",
    "import os\n",
    "def save_var(file_path):\n",
    "    dic = dict(name='Bob', age=20, score=100)\n",
    "    temp_file = pickle.dumps(dic)           # pickle.dumps函数可以将变量序列化为bytes数据\n",
    "    print(temp_file)\n",
    "    print(type(temp_file))     #\n",
    "    with open(file_path, 'wb') as file:\n",
    "        file.write((temp_file) )     # 需要将bytes数据转为字符串才能存储，只能用write函数写bytes数据\n",
    "    return \n",
    "save_file = os.path.join('.', 'LOG', 'save_file.txt')\n",
    "save_var(save_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'\\x80\\x03}q\\x00(X\\x04\\x00\\x00\\x00nameq\\x01X\\x03\\x00\\x00\\x00Bobq\\x02X\\x03\\x00\\x00\\x00ageq\\x03K\\x14X\\x05\\x00\\x00\\x00scoreq\\x04Kdu.'\n",
      "{'name': 'Bob', 'age': 20, 'score': 100}\n"
     ]
    }
   ],
   "source": [
    "def read_var(file_path):\n",
    "    with open(file_path, 'rb') as file:\n",
    "        variable = file.read()\n",
    "#         print(type(variable))\n",
    "        print((variable))\n",
    "        variable = pickle.loads(variable)\n",
    "        print(variable)\n",
    "save_file = os.path.join('.', 'LOG', 'save_file.txt')  # 该文件本身无意义，只有还原后才能看出变量\n",
    "read_var(save_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2.直接将任意对象序列化为bytes数据后写入文件\n",
    "import pickle\n",
    "def save_var2(file_path):\n",
    "    content = dict(name='Bob', age=20, score=100)\n",
    "    with open(file_path, 'wb') as file:\n",
    "        pickle.dump(content, file)      # 直接将对象序列化后存储到文件中\n",
    "save_file = os.path.join('.', 'LOG', 'save_file2.txt')\n",
    "save_var2(save_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'name': 'Bob', 'age': 20, 'score': 100}\n"
     ]
    }
   ],
   "source": [
    "#  读取pickle序列化后的数据并还原\n",
    "def read_var2(file_path):\n",
    "    with open(file_path, 'rb') as file:\n",
    "        variable = pickle.load(file)\n",
    "        print(variable)\n",
    "save_file = os.path.join('.', 'LOG', 'save_file2.txt')  # 该文件本身无意义，只有还原后才能看出变量\n",
    "read_var2(save_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "小结:\n",
    "- 使用pickle.dump(temp_data, file_path)和pickle.load(file_path)直接在数据和文件间转换，可以不用dumps和loads\n",
    "- pickle只适用于Python语言\n",
    "- file.write在文件的'wb'模式可以直接写入bytes数据，writelines只能写字符串或序列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. JSON\n",
    "对象序列化的标准格式，字符串编码是UTF-8\n",
    "\n",
    "优点：方便磁盘存储和网络传输；不同编程语言间传递对象"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.python对象转换为json对象: {\"name\": \"David\", \"age\": 20, \"score\": 100} <class 'str'>\n",
      "2.json对象转换为python对象: {'name': 'David', 'age': 20, 'score': 100} <class 'dict'>\n"
     ]
    }
   ],
   "source": [
    "# JSON和python对象相互转换\n",
    "import json\n",
    "dic = dict(name='David', age=20, score=100)\n",
    "json_out = json.dumps(dic)     # json对象本质是字符串，里面内容与python的字典类似\n",
    "print('1.python对象转换为json对象:', json_out, type(json_out))\n",
    "python_out = json.loads(json_out)\n",
    "print('2.json对象转换为python对象:', python_out, type(python_out))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'age': 20, 'name': 'David', 'score': 100}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f = open('LOG\\\\json.txt', 'w')\n",
    "json.dump(dic, f)          # 直接将python对象转换为json对象并存储到文件中\n",
    "f.close()\n",
    "f2 = open('LOG\\\\json.txt', 'r')\n",
    "js = json.load(f2)         # 直接将文件读取并转换为python对象\n",
    "f2.close()\n",
    "js"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "小结:\n",
    "- json.dumps() loads()与pickle的用法类似，首先要转换为中间的python对象，才能存到文件，dump()和load()可以直接在json数据与文本直接进行转换\n",
    "- json能够序列化类似python字典形式的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.实例的属性: {'name': 'David', 'age': 25, 'score': 100}\n",
      "2.json序列化后的类: {\"name\": \"David\", \"age\": 25, \"score\": 100}\n",
      "3.json序列化多种类: {\"name\": \"David\", \"age\": 25, \"score\": 100}\n",
      "4.json对象反序列化得到类: <__main__.Student object at 0x000001D140584710>\n"
     ]
    }
   ],
   "source": [
    "# json实例化一个类的实例\n",
    "import json\n",
    "class Student(object):\n",
    "    def __init__(self, name, age, score):\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "        self.score = score\n",
    "s = Student('David', 25, 100)       # 类不可以直接json序列化\n",
    "print('1.实例的属性:', s.__dict__)\n",
    "# 类不可以直接json序列化，先将类转换为字典再序列化，default函数将原数据处理后再序列化\n",
    "def student2dict(std):\n",
    "    return{\n",
    "        'name' : std.name,\n",
    "        'age'  : std.age,\n",
    "        'score': std.score\n",
    "    }\n",
    "out = json.dumps(s, default=student2dict)\n",
    "print('2.json序列化后的类:', out)\n",
    "# 下面的方法可适应多个类\n",
    "out1 = json.dumps(s, default= lambda obj: obj.__dict__)\n",
    "print('3.json序列化多种类:', out)\n",
    "def dict2student(d):\n",
    "    return Student(d['name'], d['age'], d['score'])\n",
    "json_out = json.loads(out, object_hook=dict2student)\n",
    "print('4.json对象反序列化得到类:', json_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "序列化练习"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.不使用ascii码: {\"name\": \"小明\", \"age\": 20}\n",
      "2.使用ascii码: {\"name\": \"\\u5c0f\\u660e\", \"age\": 20}\n"
     ]
    }
   ],
   "source": [
    "# ascii编码主要对汉语有影响，英语没有影响\n",
    "obj = dict(name='小明', age=20)\n",
    "s = json.dumps(obj, ensure_ascii=False)\n",
    "print('1.不使用ascii码:', s)\n",
    "s1 = json.dumps(obj, ensure_ascii=True)\n",
    "print('2.使用ascii码:', s1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
